Custom Report
1 Intoduction
This report is one of 493,845 that I will make, and one of 104,070,413 that could be made.
I “toke” the 1.4 TB Linked-In data that was breached in 2020, and turned it into some insights to power my job HUNT.
The insights I could share in this report, that are also related to my goals, are:
- Industry base recruitment trend.
- Company base workforce timeline.
- Current/part workforce info:
- Basic info: Name, job title, status, social link. I could add geo-location for some that have the data, but it would look creepy.
- Their work period.
- Their experiences.
2 About me
Salutations; I’m Joseph, a self-taught data analyst, engineer, and scraper.
Despite life’s challenges, my goal remains a remote job, either full or part-time, and having friends to tackle the challenges of this changing world with.
To show my skills and dedication, I made this project that yielded this tailored report.
3 About the project
3.1 How this project comes to life?
You would know by now, from my email, that I am hunting for a job.
About a year ago, I scraped contact info from Google Map to get my first job. Later I scraped contact from Linked-In website… you can check how that went in here.
Recently, I finally got to learning SQL because of DuckDB, it is a software that allows you to process big data in your local machine by using storage space as RAM; Then I remembered about a leaked Linked-In data that I couldn’t process.
Thus my journey started to learn SQL, process the data, and make something out of it.
3.2 The process
The process was done in my local machine, and it was as followed.
3.2.1 Downloaded the leaked data
I downloaded the data from a torrent.
There was around 700 .gz file, each is around 280 Mb; 196 GB in total.
Each .gz file contain a 2 GB file; 1.4 TB in total.
Each file have multiple lines, and each one of them is a JSON; Not the file is a JSON, it just have multiple JSONs, one in each line.
3.2.2 Processing the weird data
I this phase I created a script that automatically open an archive, process the file, and save it as a Parquet file with compression level of 22.
I used Python, Pathlib, Polars, and a lot of patience.
The process toke around 20 minutes per file, in total it toke around three weeks (I had to shutdown my PC at night) The result was 700 parquet files, each is around 190 Mb; 133 GB in total.
3.2.3 Making relational database
The data in the datasets were nested, especially the “experience” field, it had the experience of a person and the company info; The problem is that the company info get repeated multiple tiles, across all datasets.
Making a relational database will solve this, and make the exploratory data analysis easier.
The code was split in two:
1. I used Polars to split each of the 700 datasets into mini relational databases.
2. I used DuckDB to merge all the mini relational databases and remove duplicates in some, mainly company and university information’s.
The result was a relational database that is 73 GB in size; From 1.4 TB to 73 GB.
All of this is using my PC, so servers were harmed, only my CPU fan and my ear.
3.2.4 Filter
I filtered out companies base on their industry, country, and whether I have the email of one of the higher ups.
4 General graphs
4.1 market research indestry’s yearly new recruit count
4.2 toluna corporate’s workforce status over the years
5 Workforce sample
5.1 Adam Collins
Job title: Director of quality assurance
Associated: True
Socials: https://linkedin.com/in/adam-collins-b4535b
5.1.1 Adam Collins’s working period at toluna corporate
5.1.2 Gantt plot of Adam Collins’s experience
5.2 Adrienne Watts
Job title: Marketing manager
Associated: True
Socials: https://linkedin.com/in/adriennewatts
5.2.1 Adrienne Watts’s working period at toluna corporate
5.2.2 Gantt plot of Adrienne Watts’s experience
5.3 Aliyah Hamid
Job title: Data processor
Associated: True
Socials: https://linkedin.com/in/aliyah-hamid-51a005119
5.3.1 Aliyah Hamid’s working period at toluna corporate
5.3.2 Gantt plot of Aliyah Hamid’s experience
5.4 Andreea Rusu
Job title: Team leader - customer software support team
Associated: True
Socials: https://linkedin.com/in/andreea-oana-rusu-04900b18
5.4.1 Andreea Rusu’s working period at toluna corporate
5.4.2 Gantt plot of Andreea Rusu’s experience
5.5 Bob Leitman
Job title: Regional vice president, health care
Associated: True
Socials: https://linkedin.com/in/bob-leitman-baa68513
5.5.1 Bob Leitman’s working period at toluna corporate
5.5.2 Gantt plot of Bob Leitman’s experience
5.6 Christelle Alexandre-Vincent
Job title: Key account director
Associated: True
Socials: https://linkedin.com/in/christelle-alexandre-vincent-09130030 | https://linkedin.com/in/christellealexandrevincent
5.6.1 Christelle Alexandre-Vincent’s working period at toluna corporate
5.6.2 Gantt plot of Christelle Alexandre-Vincent’s experience
5.7 Cindy Lemos
Job title: Head of global audience segmentation
Associated: True
Socials: https://linkedin.com/in/cindy-lemos-b6a4441 | https://linkedin.com/in/cindylemos
5.7.1 Cindy Lemos’s working period at toluna corporate
5.7.2 Gantt plot of Cindy Lemos’s experience
5.8 Cristian Cutrona
Job title: Senior business development manager
Associated: True
Socials: https://linkedin.com/in/cristian-cutrona-a056753
5.8.1 Cristian Cutrona’s working period at toluna corporate
5.8.2 Gantt plot of Cristian Cutrona’s experience
5.9 Daniel Kuppersmith
Job title: Client development associate
Associated: False
Socials: https://facebook.com/daniel.kuppersmith | https://linkedin.com/in/daniel-kuppersmith-16774
5.9.1 Daniel Kuppersmith’s working period at toluna corporate
5.9.2 Gantt plot of Daniel Kuppersmith’s experience
5.10 Donna Hentges
Job title: Panel services manager
Associated: False
Socials: https://linkedin.com/in/donna-hentges-475966 | https://twitter.com/donnaenticott
5.10.1 Donna Hentges’s working period at toluna corporate
5.10.2 Gantt plot of Donna Hentges’s experience
5.11 Fabio Caiano
Job title: Lavoro da casa
Associated: True
Socials: https://linkedin.com/in/fabio-caiano-77037137
5.11.1 Fabio Caiano’s working period at toluna corporate
5.11.2 Gantt plot of Fabio Caiano’s experience
5.12 George Terhanian
Job title: Group chief strategy and products officer; north american president
Associated: False
Socials: https://twitter.com/georgeterhanian | https://plus.google.com/+georgeterhanian | https://linkedin.com/in/george-terhanian-087a641
5.12.1 George Terhanian’s working period at toluna corporate
5.12.2 Gantt plot of George Terhanian’s experience
5.13 Gian Saleri
Job title: Marketing director, south europe and latam
Associated: True
Socials: https://linkedin.com/in/gian-marco-saleri-58577624 | https://twitter.com/gianmarcosaleri | https://linkedin.com/in/gmsaleri
5.13.1 Gian Saleri’s working period at toluna corporate
5.13.2 Gantt plot of Gian Saleri’s experience
5.14 Hunny Arora
Job title: Senior executive - finance
Associated: True
Socials: https://twitter.com/hunny_arora | https://linkedin.com/in/hunny-arora-b5844b96
5.14.1 Hunny Arora’s working period at toluna corporate
5.14.2 Gantt plot of Hunny Arora’s experience
5.15 James Parent
Job title: Senior director; client development
Associated: True
Socials: https://facebook.com/james.parent2 | https://linkedin.com/in/james-parent-2012a89 | https://linkedin.com/in/james-parent-31a42a7
5.15.1 James Parent’s working period at toluna corporate
5.15.2 Gantt plot of James Parent’s experience
5.16 Jannat-Ul-Erum ️����������
Job title: Surveyor
Associated: True
Socials: https://linkedin.com/in/jannat-ul-erum-🕋👑👸⚓👩🎓🏡🚗✈️😇👨👩👦👦-60584171
5.16.1 Jannat-Ul-Erum ️����������’s working period at toluna corporate
5.16.2 Gantt plot of Jannat-Ul-Erum ️����������’s experience
5.17 Jay-Ar Carmelotes
Job title: Affiliate promoter of toluna
Associated: True
Socials: https://linkedin.com/in/jay-ar-carmelotes-b243b4162
5.17.1 Jay-Ar Carmelotes’s working period at toluna corporate
5.17.2 Gantt plot of Jay-Ar Carmelotes’s experience
5.18 Jeff Inglis
Job title: Svp, enterprise sales
Associated: True
Socials: https://linkedin.com/in/jeff-inglis-7156961b | https://linkedin.com/in/jeffreyinglis
5.18.1 Jeff Inglis’s working period at toluna corporate
5.18.2 Gantt plot of Jeff Inglis’s experience
5.19 Jennifer Weitz
Job title: Vice president, global supply
Associated: False
Socials: https://linkedin.com/in/jennifer-weitz-29a8611
5.19.1 Jennifer Weitz’s working period at toluna corporate
5.19.2 Gantt plot of Jennifer Weitz’s experience
5.20 Jim Mullady
Job title: Client director
Associated: True
Socials: https://linkedin.com/in/jim-mullady-8a372647
5.20.1 Jim Mullady’s working period at toluna corporate
5.20.2 Gantt plot of Jim Mullady’s experience
5.21 John Coffey
Job title: Senior director, client development
Associated: True
Socials: https://linkedin.com/in/john-coffey-b4237013
5.21.1 John Coffey’s working period at toluna corporate
5.21.2 Gantt plot of John Coffey’s experience
5.22 John Maiolino
Job title: Respondent
Associated: False
Socials: https://linkedin.com/in/johnmaiolino | https://linkedin.com/in/john-maiolino-8213b083 | https://linkedin.com/in/john-maiolino
5.22.1 John Maiolino’s working period at toluna corporate
5.22.2 Gantt plot of John Maiolino’s experience
5.23 Lavi Avigdor
Job title: Team lead
Associated: True
Socials: https://foursquare.com/user/24510192 | https://crunchbase.com/person/lavi-avigdor | https://linkedin.com/in/avigdor | https://klout.com/laviavigdor | https://stackoverflow.com/users/483647 | https://github.com/laviavigdor | https://twitter.com/laviavigdor
5.23.1 Lavi Avigdor’s working period at toluna corporate
5.23.2 Gantt plot of Lavi Avigdor’s experience
5.24 Marco Pasqualina
Job title: Head of sales, north america
Associated: True
Socials: https://linkedin.com/in/marco-pasqualina-654565 | https://linkedin.com/in/marcopasqualina
5.24.1 Marco Pasqualina’s working period at toluna corporate
5.24.2 Gantt plot of Marco Pasqualina’s experience
5.25 Mark Shepard
Job title: Director, japan
Associated: True
Socials: https://linkedin.com/in/mark-shepard-55382
5.25.1 Mark Shepard’s working period at toluna corporate
5.25.2 Gantt plot of Mark Shepard’s experience
5.26 Matt Mellinger
Job title: Client associate
Associated: True
Socials: https://twitter.com/matt_mellinger | https://linkedin.com/in/matthewmellinger | https://linkedin.com/in/matt-mellinger-4b726726 | https://facebook.com/mattmell
5.26.1 Matt Mellinger’s working period at toluna corporate
5.26.2 Gantt plot of Matt Mellinger’s experience
5.27 Meetali Tewari
Job title: Data analyst
Associated: False
Socials: https://linkedin.com/in/meetali-tewari-6408032a
5.27.1 Meetali Tewari’s working period at toluna corporate
5.27.2 Gantt plot of Meetali Tewari’s experience
5.28 Michael Dawson
Job title: Svp, strategic accounts
Associated: True
Socials: https://linkedin.com/in/michael-dawson-3430114
5.28.1 Michael Dawson’s working period at toluna corporate
5.28.2 Gantt plot of Michael Dawson’s experience
5.29 Michele Morelli
Job title: Svp, global marketing strategy
Associated: True
Socials: https://linkedin.com/in/michele-morelli-52401023 | https://linkedin.com/in/michelexmorelli
5.29.1 Michele Morelli’s working period at toluna corporate
5.29.2 Gantt plot of Michele Morelli’s experience
5.30 Neil Dixit
Job title: Vice president digital products and director of product technology
Associated: False
Socials: https://angel.co/neil-dixit | https://gravatar.com/neilmdixit | https://myspace.com/neildixit | https://twitter.com/neildixit | https://facebook.com/neil.dixit | https://klout.com/neildixit | https://linkedin.com/in/neil-dixit-4420464 | https://linkedin.com/in/neildixit
5.30.1 Neil Dixit’s working period at toluna corporate
5.30.2 Gantt plot of Neil Dixit’s experience
5.31 Ofir Shilovski
Job title: Team leader
Associated: True
Socials: https://linkedin.com/in/ofir-shilovski-5805191
5.31.1 Ofir Shilovski’s working period at toluna corporate
5.31.2 Gantt plot of Ofir Shilovski’s experience
5.32 Patricia Chapin-Bayley
Job title: Group svp, media and entertainment
Associated: True
Socials: https://linkedin.com/in/patricia-chapin-bayley-1a47783
5.32.1 Patricia Chapin-Bayley’s working period at toluna corporate
5.32.2 Gantt plot of Patricia Chapin-Bayley’s experience
5.33 Phil Ahad
Job title: Chief digital officer
Associated: True
Socials: https://youtube.com/user/pahad210 | https://linkedin.com/in/philahad
5.33.1 Phil Ahad’s working period at toluna corporate
5.33.2 Gantt plot of Phil Ahad’s experience
5.34 Rachana Randev
Job title: Project manager
Associated: False
Socials: https://linkedin.com/in/rachana-randev-4969277
5.34.1 Rachana Randev’s working period at toluna corporate
5.34.2 Gantt plot of Rachana Randev’s experience
5.35 Renato Rapucci
Job title: Associate director client services
Associated: True
Socials: https://linkedin.com/in/renato-rapucci-2871b26
5.35.1 Renato Rapucci’s working period at toluna corporate
5.35.2 Gantt plot of Renato Rapucci’s experience
5.36 Ruba Ayyat
Job title: Senior client service manager, enterprise services
Associated: True
Socials: https://linkedin.com/in/ruba-ayyat-mba-04684a11 | https://linkedin.com/in/rubaayyat
5.36.1 Ruba Ayyat’s working period at toluna corporate
5.36.2 Gantt plot of Ruba Ayyat’s experience
5.37 Ryan Lewis
Job title: Director of product, audience and targeting
Associated: True
Socials: https://linkedin.com/in/ryan-lewis-5a018612
5.37.1 Ryan Lewis’s working period at toluna corporate
5.37.2 Gantt plot of Ryan Lewis’s experience
5.38 Sarit Zeidman
Job title: .net full stack web developer
Associated: True
Socials: https://linkedin.com/in/sarit-zeidman-19218b1 | https://twitter.com/saritzed | https://facebook.com/sarit.zeidman
5.38.1 Sarit Zeidman’s working period at toluna corporate
5.38.2 Gantt plot of Sarit Zeidman’s experience
5.39 Tejal Samant
Job title: Senior manager, digital services and communities
Associated: False
Socials: https://twitter.com/tssamant | https://linkedin.com/in/tejalsamant
5.39.1 Tejal Samant’s working period at toluna corporate
5.39.2 Gantt plot of Tejal Samant’s experience
5.40 Tim Kunkel
Job title: Manager: business development
Associated: False
Socials: https://linkedin.com/in/timkunkel | https://twitter.com/_timik_
5.40.1 Tim Kunkel’s working period at toluna corporate
5.40.2 Gantt plot of Tim Kunkel’s experience
5.41 Tsahi Yosef
Job title: Senior vice president product innovation
Associated: True
Socials: https://linkedin.com/in/tsahiby | https://linkedin.com/in/tsahi-ben-yosef-16185b4